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Abstract 

In this paper, we study causal discrete-memoryless relay networks (DMRNs). The network consists 
of multiple nodes, each of which can be a source, relay, and/or destination. In the network, there are 
two types of relays, i.e., relays with one sample delay (strictly causal) and relays without delay (causal) 
whose transmit signal depends not only on the past received symbols but also on the current received 
symbol. For this network, we derive two new cut-set bounds, one when the causal relays have their own 
messages and the other when not. Using examples of a causal vector Gaussian two-way relay channel 
and a causal vector Gaussian relay channel, we show that the new cut-set bounds can be achieved by 
a simple amplify-and-forward type relaying. Our result for the causal relay channel strengthens the 
previously known capacity result for the same channel by El Gamal, Hassanpour, and Mammen. 

I. Introduction 

A model of relay networks was first introduced by van der Meulen in [1], [2]. In [3], Cover and 
El Gamal provided the cut-set bound for the relay channel and introduced two coding strategies, 
decode-and-forward and compress-and-forward. The coding strategies can achieve the capacities 
of some relay channels. For more complicated networks, a general cut-set bound is provided in 
[4]. In these relay networks, it is usually assumed that the relay's operation is strictly causal, i.e., 
its transmit symbol depends only on its past received symbols. Recently, new types of relays 
were introduced in [5], namely causal and noncausal relays. In the causal relay channel, the 
relay's transmit symbol depends on both the past and the current received symbols and in the 
noncausal relay channel, the relay's transmit symbol can depend on future received symbols 

The material in this paper was presented in part at the Allerton Conference on Communication, Control, and Computing, 
Monticello, IL, USA, Sep. 2011. 
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as well. In [6], Wang and Naghshvar provided an improved upper bound for the noncausal 
relay channels. For the interference channel with a causal relay, Chang and Chung established 
outer bounds and also showed that they can be achieved if the relay's power exceeds a certain 
threshold and some additional conditions are satisfied [7]. In [8], cut-set bounds were established 
for DMRNs without ordering among casual relays. Also, we showed cut-set bounds for DMRNs 
with causal side information in [9]. Recently, Fong showed a cut-set bound for generalized 
networks including both strictly causal and causal relays in [10]. Also, Kramer developed a 
cut-set bound for a network with memory inside each block of symbols in [11]. 

In this paper, we focus on the DMRN with multiple sources and destinations. We assume 
each relay node is either causal or strictly causal. We show cut-set outer bounds for two types 
of networks, one with own messages at causal relays and the other without. Our bounds reduce 
to the classical cut-set bound [4] if there is no causal relay in the network. We provide examples 
of simple causal DMRN's such as the causal vector Gaussian two-way relay channel (TWRC) 
and the causal vector Gaussian relay channel and show a simple amplify-and-forward relaying 
can be optimal if the relay's power exceeds a certain threshold. Our result for the causal scalar 
Gaussian relay channel strengthens the previous capacity result for the same channel in [5] by 
removing a constraint in Proposition 9 in [5]. 

The remainder of this paper is organized as follows. In Section II, our network model is 
introduced. In Section III, we provide two new cut-set bounds for causal DMRN's and compare 
the bounds with previously known results and then, in Section IV, we give examples of causal 
DMRN's. Finally, we conclude the paper in Section V. 

II. Model 

A multiple- source multiple-destination causal DMRN of K nodes 

W>\ \ 

x x x ... x x K , JJj?(j/j|xatu &[i:j-i] s y[i-.j-i])p(yAfi\ x M , xjvi, yjv a ),yi x ... x y K \ 

i=l / 

consists of sender alphabets X k and receiver alphabets y k , k G [1 : K] = {1, 2, . . . , K} and a 
conditional pmf 

Y[p{yj\x/ji, z[iy_i], y[i:j-i])p{yM.\xff , %ati, VN ), (!) 

3=1 
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where Ao = {1,2,..., |Ao|} C [1 : K] is the set of causal relays, and Ai — [1 : A'] \Ao is the set 
of strictly causal relays. Notations such as xj^ denotes {x k \k G Ao}. Note that the causal relays 
are ordered to avoid infinite loops among causal relays. A (2 nRl1 , 2 nRl2 , . . . , 2 nRliK , n) code for 
this network consists of K 2 message sets, K encoders, and K decoders. Encoder k G Ao assigns 
an input symbol x k i to each (m k i, . . . , m k K, yl), and encoder k E Mi assigns an input symbol 
Xki to each (m k i, ■ ■ ■ ,m kK ,ylT l ) at time i G [1 : n], where m k j is the message from node k 
to node j, and decoder k G [1 : K] assigns a set of message estimates (fh lk , . . . , rh Kk ) to each 
received sequence y%, where rhj k is a decoded message from nodes j to k. We assume R kk = 
for all k G [1 : K]. 

We also consider three special cases. The first is a causal DMRN without messages at causal 
relays, i.e., encoder k G Ao assigns an input symbol x ki to each y k for i G [1 : n\. The other 
two are causal DMRNs with/without messages at causal relays with a conditional pmf 

PiVNo I Wi )viVMi I XN , , VMo ) j ( 2 ) 

i.e., the received signals of causal relays depend on the transmit signals of the strictly causal 
relays only. 

III. Main results 

In this section, we derive new cut-set bounds for the scenarios we consider. First, we show a 
cut-set bound for the causal DMRN, where each node in the network has its own messages, and 
then show a cut-set bound for the causal DMRN without messages at causal relays. For each 
scenario, we also consider the channel given as (2) as special cases. 
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A. Causal DMRN with messages at causal relays (general case) 

We derive a cut-set bound for the causal DMRN when every node has messages to send. 
Theorem 1: If the rates are achievable in the causal DMRN with a conditional pmf given 
as (1), there exists some joint probability distribution 

pO&m) II {K%^M>z[iy-i],y[i : j-i])K^^ 

such that 



^2 R i k 

jes,k<=s c 

\u c \ ( v v 

< ^2 I J V ' A [Wi-l]\{'i»la.».,f I --i}' . 
j=l \ ^l^j-llUiLla,...,^-!} 



-^v c , , ^ 2 ) ■ • ? -^Jj-i ) | (3) 

^1) ^2 3 • • ■ 3 



+ I(X s ,Y u ;Y V c\Y Uc ,Xso), 
for all S C [1 : K], where U = S r\ M , U c = N \U = {l u . . k < lj if i < j, 

V = 5nM, and V c = M \ V. 

Proof: See Appendix A. ■ 
As a special case of a conditional pmf given as (2), we get the following cut-set bound. 
Corollary 1: If the rates Rj k are achievable in the causal DMRN with a conditional pmf given 

as (2), there exists some joint probability distribution 

Ml 

pO*Wi) II {K%-| x Mj2f[iy-i])p(ajl^ (4) 
i=i 

such that 

i^-fc < ^/(X v ,Y [1% _ 1]UZl! ...^_ l} ;^ i |Xvc,^ 1 ,...,^._ 1 ) 
jes,fce<s c j=i (5) 

+ i(x s ,y w ;y V c|F W c,x S c), 

for all 5 C [1 : If], where U = S r\ U c = N \U = {h, l t < lj if i < j, 

V = SnA/i, and V c = M \ V. 
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Proof: The cut-set bound (3) becomes 

^V,^[l^-l]\{«i,...^_i}, 



E R fsE' 

j€S,kes c j=i 



(a) 



^l^-l]^!,...,/^!} 

f /(x 5 ,y M; y v «l^,^) 



l« c l 

E 



+ /(x <s ,y w; y vc |y W c,x <sc 

|W C 



; y. 





/ 


Aye, 




f 


Xy,Xy c , 


1 


< H 




• • • 5 ^Ij-H 


- H 






> 






Yh, ■ ■ ■ ,Yij-! j 




\ 


Y { l: h -1] J 


* 



<J2{H(Y h \X Vc ,Y h ,...,Y h _ 1 )-H\Y lj 

3=1 



Xy, Xyc, 
^[Wj-l], ^[1^-1] 



(6) 



+ I(Xs, Y u \ Y V c\Y U c,X S c) 
\u c \ 

Y,{H(Y h \x V o, Y h ,..., y,_j - if (y 3 |x v , x V c, y [w ,_i])} 



+ ^(^5 ; y«; yv c iy/ c 5 -^5 c ) 
= (^v,^ / [i:« J -i]\{« 1 ,...,/ J _ 1 };y J i^v c ,yi, ■ ■ • 

+ I(X s ,Y u ;Y V c\Y U c,X S c), 

where (a) is because conditioning reduces entropy, (b) is because of Markov chain 
X[i-.ij- 1] ~~ (^v> -Xyc, y[i:Zj-i]) - ► y r Finally, we get the cut-set bound equivalent to the bound 
(5). ■ 
Proposition 1: The cut-set bound in Corollary 1 coincides with the classical cut-set bound 
[4] if A/q = 0. 

Proof: If Ao = 0, the probability distribution (4) becomes 
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Then, the mutual information in (5) becomes 



/] Rjk < I(Xs]Y V c\X S c) 

jes,kes c 



I(Xs] Y S c\X S c), 



which is same as the classical cut-set bound. 



B. Causal DMRN without messages at causal relays 

In this subsection, we consider the causal DMRN in which the causal relays do not have their 
own messages. Under the restriction, we show a tighter bound in this subsection than the one 
in the previous subsection. 

Theorem 2: If the rates Rj k are achievable in the causal DMRN without messages at causal 
relays with a conditional pmf given as (1), there exists some joint probability distribution 



p{u Afo ,X Afl ) Y[p{Vj\ x Jfi> x [l:j-lb y[l-j-l])p(yAfi\ x Aro, VNuVMo) 
3=1 

and x k = x k (y k , u k ) for k 6 Aq such that 



(6) 



jev,kes c j=i 



^V,^[l^-l]\{ii,...,i,-i}> 
Y\v.i ] ~\]\{i u ...,ij-i\ 



\ 



Xin ■ ■ ■ j 



\ 



Yi 



(7) 



for all S C [1 : K], where U = S n Af , U c = Af \U = {h, l\u-\}, U < lj if i < j, 
V = 5nM, and V c = M \ V. 

Proof: See Appendix B. ■ 

As a special case of a conditional pmf given as (2), we get the following cut-set bound. 

Corollary 2: If the rates Rj k are achievable in the causal DMRN without messages at causal 
relays with a conditional pmf given as (2), there exists some joint probability distribution 



P(UM > x Ni ) \\ PiVj I x Afi , V[l:j-1] )P{VN 1 1 X N , x Afi , VMo ) 
3=1 



(8) 
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and x k = x k (y k , u k ) for k E A/q such that 



E %<E J (* 



-i}j^jl^w c )^v c )^/u • • • >Y ij _ lj 



jev,k€S c j=i (9) 

+ I{Xv, Uu] Y V c\U U c ,Y U c , X V c) , 
for all S C [1 : K], where W = 5 n Af , U c = Af \ U = {h, . . . , l m }, l t < l 3 - if i < j, 
V = SnM, and V c = M \ V. 

Proof: Because yj^ depends only on xjvi, the cut-set bound (7) becomes 

( u x ^ 



|W C | 

E ^<E J 

jes,kes c j=i 



\ 



Y[id j -i]\{h,...,i j -i} 



+ I(Xv, U u ; Y V c\Uuc,Y U c, X\ 



\w\ 

E 



r / 



H 



{ \ 



Xh, ■ ■ ■ j ^i-i 



H Yu 



Uuc, Xy, X V c, 

Xii-^-i], 



Yi t ,. . . , Yi._ x 



H Yu 



Uu c ,X V , Xyc, 
^[1:1,-1] j^Wj-l] 



(6) 



IWI 

El^J^^V^^^.-.T^J-^^J^^v,^,^!:^!])} 



E^V^l: 



-i}',Yi j \Uuo,X V c,Y h , . . . jYi^] 



where (a) is because conditioning reduces entropy, (b) is because of Markov chain 

X[\.ij-i] (Uu c , X v , X V c, Y[i : ;._i]) — >■ yj.. Finally, we get the cut-set bound equivalent to the 

bound (9). ■ 
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C. Comparison of two cut-set bounds 

Proposition 2: If there are no messages at the causal relays, the cut-set bound for the causal 
DMRN shown in Theorem 1 includes the cut-set bound for the causal DMRN without messages 
at causal relays shown in Theorem 2. 

Proof: Assuming the probability distribution (6), we get the following inequality. 



|w c l 

E' 



/ 



X^^X^ij-xWih,...,^}, 



Yu 



Y 



\ 



[Uj-iMh,...,^} 



Uu c ,X V c, 
^h, ■ ■ ■ , X\._^ 
Y h , . . . J 



+ I{Xv, Uu\ Yvc\Uuc,Yuc, X\ 



E 



/ 



H 



Yu 



V 



Uu c ,X V c, 
Xi i: . . • , Xi._ i: 



/ 



H 



) 



Yu 



\ 



Uu c ,X V c, X v , 



\ 1 



Y[i aj -i] J t 



+ H(Y V c\Uuc,Y U c, X V c) — H(Y V c\Uuc,Y U c, X V c, X v , Uu) 



(a) 



\W\ 

E 



I 



H 



Yu 



u x ^ f 



H 



Y, 



X V c,X v , 
Xii-^-i], 

¥[1:1,-1] 



\ 1 



/ J 



iJ(Yy c |^W c , H/c, Xyc, X W c) — //(Y"v c |^W c , ¥^c, A^c, X\>, Uu) 



§5 



/ 



V 



-^ii) • • • i 



\ 



/ 



V 



\ 1 



11 



[!:{,-!] / J 



+ H(Yvc\Yuc,Xyc, X U c) — H(Yvc\Uu°, Yu^,Xs^, X v , Uu, Yt 



u 



/ 



H 



Y u 



\ 



X \ ( 

Aye, 

Xi ± , ■ ■ ■ jXij^, 
Y h , . . . ,Y h _ x J \ 



H 



Xyo, Xy, 

X\l:U-l], 



Y 



[1:1,-1] ) J 



+ H(Y V c\Yuc, X V c, X U c) — H(Y V c\Uuc,Yu c ,Xsc,Xs, Uu, Yi 



u 
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V 



^Zi, ■ ■ • , Yi. 



\ 



H 



ij-i 



j'=i 



7 V 7 J 

Ave, 

■^Vj^I^— l]\{«i,Ja,...,ij-i}! 



V 



y[l:Z 3 -l]\{Zi,Z 2> ...,Z 3 -i} 



+ /(x < s,y w ;y vc |y wc ,x 5c 



where (a) is because A^c is a function of l^ c ) an d a Markov chain 
Uuc—t- (X v , Xy Cj X[i : i i], Y[i : i—i]) — > Y t ., (6) is because conditioning reduces entropy, (c) is 
because Xu is a function of (Uu,Yu), and (d) is because of the Markov chain {Uu c ^u)^ 
(Yuc, X V c, X v , X U c,Y u , X u ) — > Y V c ■ 
In some channels, the cut-set bound in Theorem 1 is strictly larger than the cut-set bound in 
Theorem 2. Consider the following three-node relay channel, where Ao = {2} and J\f\ = {1,3}. 

Y 2 = (Xt + Z 2 ) mod 4 
Y 3 = Y 2 mod 2, 

where Z 2 is Bem(|) and is independent of X±, and the cardinality of X\ is 4. For this channel, 
Theorem 1 becomes 



max min{J(A"i; Y 2 ) 

p(xi)p(x2\xi,y2) 



J(X i; Y 3 \X 2 , Y 2 ),I{X 1 ,X 2 , Y 2 - Y 3 )} 



max mm{H(Y 2 ) - 1, #(T 3 )} 

p(xi)p(x2|a;i,j/2) 
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and Theorem 2 becomes 



max 

p{u 2 ,Xl),X 2 {u 2 ,y 2 ) 



min{J(X i; Y 2 , Y 3 \U 2 ), I(X U U 2 ; Y 3 )} 



< 



max 

P(U2 , XI ), I2(u2,3/2) 



I(X U U 2 ;Y 3 ) 



max 

p(ll2,Xl),X2(li2,2/2) 



^(^3) 



H(Y 3 \X 1} U 2 ) 



(a) 



H{Y 3 ) 



H{Z 2 \X X ,U 2 ) 



max 

p(«2,xi),x 2 (1x2,2/2) 



max 

p(u 2 ,x 1 ),x 2 (u 2 ,y 2 ) 



H(Y 3 ) 



H(Z 2 ) 



max 

p(lt 2 , Xi ),X 2 («2,J/2) 



1 



where (a) is because F 3 = (Xl + Z 2 ) mod 2. Thus, the rate region in Theorem 1 can be strictly 
larger than that of Theorem 2. However, Theorem 1 can still be useful since it can be used to 
prove some capacity results as in Section IV. 

IV. Examples 

In this section, we use a causal vector Gaussian TWRC and a causal vector Gaussian relay 
channel as examples of causal DMRN's. We apply the cut-set bounds in Theorems 1 and 2 for 
the channels and show the bounds can be tight under some conditions. 

A. Causal vector Gaussian TWRC 

In a causal TWRC, nodes 1 and 3 exchange their messages with help of causal relay node 
2. Let X k denote the transmit signal and Y k denote the received signal of node k 6 [1 : 3]. For 
this channel, Ao = {2} and M\ = {1,3}. Then, the cut-set bound in Theorem 1 reduces to 




(10) 



for some p{x 1 ,x 3 )p{x 2 \x u x 3l y 2 ) . 
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Fig. 2. Vector Gaussian TWRC with causal relay 2 



For the causal vector Gaussian TWRC as shown in Fig. 2, the received signal at each node is 

Yi = H12X2 + H13X3 + Zi 
Y 2 = H21X1 + H23X3 + Z 2 
Y3 = H32X2 + H31X1 + Z3, 

where Hjk E C TjXtk is the channel gain from nodes k to j, % is the number of transmit 
antennas of node k and rj is the number of receive antennas of node j, tr(E[XfcXj,]) < and 
Zj ~ CAf(0, 1) for A; = 1,2,3 and j = 1, 2, 3. Let S fc = E[X fc x£] for k = 1, 2, 3. 

Theorem 3: If the transmit power of node 2 satisfies tr[F(H 2 i £^ H\ x + H 23 H^ 3 +I)F f ] < 
P 2 , the capacity region of the causal vector Gaussian TWRC is 



R 13 < max loglAaVgSxVaAa 

tr(Si)<Pj 



II 



R 31 < max log \A 1 \\j: 3 V 1 A 1 + I|, 

tr(S 3 )<P 3 



where F 



U 31 H 32 
U f 13 H 12 



U 



, E* and £3 are chosen among values satisfying 

E* = argmax tr(Ei) < Pi log lAgV^SiVgAg + I| 



S* = argmax tr(E3) < P;! 



loglAjVjEgViAi + II, 



U21, U31, U23, U13, Ai, and A 3 are obtained by singular value decomposition (SVD) of channel 
matrices, i.e., 



H21 




U21 


H31 




U31 



A,V 



3 V 3> 
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H 2 3 




u 23 


H13 




U13 



AiVj, 

-H-13 U13 

and (A)7 X = A^AA 1 ") -1 denotes the right inverse of A. Here, [U^U^f, [U^Ufgf, Vi, and 
V 3 are unitary matrices, and A 1 and A 3 are diagonal matrices with positive entries. 
Proof: 

Achievability. Assume that F is chosen as in the theorem. Let nodes 1 and 3 transmit 
codewords using independent vector Gaussian codebooks with covariance matrices and Eg, 
respectively. We apply AF relaying using F, i.e., the relay's transmit signal X 2 is formed by 
X 2 = F Y 2 . Because X 2 satisfies the power constraint from the condition given in the theorem, 
it can be transmitted from the relay. Then, the received signal at node 1 is 

Yi = H12 X 2 + H13 X3 + Zi 
= H 12 F Y 2 + H 13 X 3 + Zi 
= H 12 F(H 23 X 3 + H 21 Xi + Z 2 ) + H 13 X 3 + Z 1 
= (H12 F H 2 3 + H13) X3 + Hi 2 F H 2 i Xi + Hi 2 F Z 2 + Zi . 
Using the knowledge of X x at node 1, the node can calculate Yi defined as follows: 
— U{ 3 (Y! — H 12 F H 2 i XO 

= U f 13 (H 12 FU 23 + U 13 )A 1 Vl X 3 +U f 13 (H 12 F Z 2 + Z x ) 
= (U 23 U 23 + U t 13 U 13 )AiVl X 3 +U 23 Z 2 +U f 13 Z 1 



(6) 



A 1 y\x 3 +z 1 , 



where (a) is because U| 3 H 12 F = from the choice of F 



U 31 H 32 


-1 


u 21 


U f 13 H 12 







is because [U^ 3 Uf 3 ] T is a unitary matrix, and Z x = \j\ 3 Z 2 +U{ 3 Z x with E[ZiZ{] = I. The 
covariance matrix of Yi is 



(6) 



E[YxY;] = AiVlS^VxAx + I. 
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Then, R 31 up to the following is achievable: 

7(X 3 ;Y 1 ) = ^(Y 1 )- J ff(Y 1 |X 3 ) 

= H(Y 1 )-H(Z 1 ) 

= log|A 1 v;s*V 1 A 1 + I|. 

Similarly, Ri 3 up to the following is achievable: 

J(Xi; Y 3 ) = log |A 3 V 3 ^V 3 A 3 + I| . 

Converse. Applying the cut-set bound (10) to the causal vector Gaussian TRWC, the first 
term in the bound for i? 13 in (10) can be written as follows: 

J(X i; Y 2 1 X 3 ) + 7(X i; Y 3 1 X 2 , X 3 , Y 2 ) 
= / 1 (Y 2 |X 3 )-/i(Y 2 |X 3 ,X 1 ) 
+ h(Y 3 I Y 2 , X 3 , X 2 ) - h(Y 3 I Y 2 , X 3 , X 2 , Xx) 

< h(Y' 2 ) - h(Y' 2 I Xx) + h(Y' 3 I Y' 2 , X 3 , X 2 ) 
— h(Y 3 I Y 2 , X 3 , X 2 , Xi) 

(b) 

< M Y 2 ) - h(Y f 2 I Xi) + h(Y 3 I Y 2 ) 
-^(Y^Y^Xa.Xa.Xj) 

( = } h(Y' 2 ) - h(Y' 2 I X x ) + h(Y' 3 I Y' 2 ) - h(Y 3 I Y 2 , X 1 ) 

= /(X i; Y 2 ,Y 3 ) 

= /i(Y 2 ,Y 3 )-/i(Z 2) Z 3 ) 

< log(Tre) J J _ log^e)^ 3 ) 

= logical 



July 19, 2012 



DRAFT 



14 



log 

log 
log 
log 



E 



Y' 

1 2 

Y' 



[Y? Y 3 



'ti 



[ H 21 H 3i] Ts i( H 2i H ; 



AaVjSiVsAa 



< max 

tr(£i)<Pi 



AaVjSiVsAs + 1 



where Y' 2 = H 2 i X x + Z 2 , Y' 3 = H 3 i X 1 + Z 3 , Y 3 = [Y 2 T Y 3 T ] T , (a) and (6) are because 
conditioning reduces entropy, and (c) is because of the Markov chain (X 2 ,X 3 ) — > (Xi,Y 2 ) — > 



Y 3 - 



Similarly, we get an upper bound on R 31 . Thus, the rates are upper bounded as 



R 13 < max log |A 3 v5SiV 3 A 3 

tr(£i)<Pi 

R 31 < max loglAiVjSgVxA! 

tr(S 3 )<P3 



Since, the achievable rate region and the outer bound coincide, the proof is completed. ■ 
As a simpler example of a causal vector Gaussian TWRC, assume — 1 for k — 1, 2, 3, 
ti = t 3 = 1, and t 2 = 2. The received signal at each node is 

Y\ = hnX 2 \ + ^13^3 + Z\ 

Y 2 = h 2 \Xi + h 23 X 3 + Z 2 

Y 3 = h 32 X 23 + hs\Xi + Z 3 , 

where hjy, is the channel gain from nodes k to j, H12 = [hi 2 0], H 32 = [0 h 32 ], and X 2 = 



[X 2 i X 23 ] T . If P 2 > 



|fcas| : 



+ 



l^2l| 2 



\h 12 \ 2 \h 13 \ 2 ^ \h S 2\ 2 \h ai \ 



achieved by choosing F 



(\h 2 i\ 2 Pi + \h 23 \ 2 P 3 + 1), then the capacity can be 



^12^13 /l32^3i 



B. Causal vector Gaussian relay channel 

In this subsection, we consider a causal relay channel, where node 1 transmit its message to 
node 3 with help of a causal relay node 2, as a special case of a causal TWRC. For this channel, 
the cut-set bound of Theorem 2 reduces to 



R 13 < max min{/(X 1 ; Y 2 , Y 3 1 U 2 ) , I{X 1 , U 2 ; Y 3 ) } , 



(ID 
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R13 < max min 

p(x 1 ) P (x 2 \x 1 ,y 2 ) I(Xi,X 2 , Y 2 ; Y 3 



(12) 



where the maximization is over p(u 2 , £1) and x 2 = x 2 {u 2l y 2 ). This recovers the upper bound in 
Theorem 2 in [5]. If we use Theorem 1 instead, then we get a potentially looser bound given as 

I(X 1 ;Y 2 ) + I(X 1 ;Y 3 \X 2 ,Y 2 ), 

Note that this is the same as the result in Theorem 1 of [5] (if modified, see the footnote below), 
where they showed an upper bound for the non-causal relay channel with three nodes. 1 
For the causal vector Gaussian relay channel, the received signal at each node is 

Y2 = H21 Xi + Z2 

Y3 = H31 Xi + H32 x 2 + z 3 , 

where Hjk E Qn^k j s me channel gain from nodes k to j, is the number of transmit 
antennas of node k and r, is the number of receive antennas of node j, tr(E[X fe X[]) < P k and 
Zj ~ CAf(0, 1) for k=l,2 and j = 2, 3. Let S fc = E[X fc Xj] for k = 1, 2. For this channel, we 
can directly obtain the following from Theorem 3. 

Proposition 3: If the transmit power of node 2 satisfies tr[F(H 2 i +I)F^] < P 2 , the 

capacity region of the causal vector Gaussian relay channel is 



R 13 < max loglAV^xVA + II, 

tr(Ei)<Pi 



where F = (U^H^),. l ~U 2 \, S* is chosen among values satisfying 

S* = argmax tr(Ei) < Pi log {AV^VA + I|, 

U2i,U3i, A and V are obtained by singular value decomposition (SVD) of channel matrices, 
i.e., 



AV f . 



Here, [U^U^] 7 , and V are unitary matrices, and A is diagonal matrix with positive entries. 



H21 




U21 


H31 




U31 



'Note that the supremum in (7) in [5] should be over p(xi)p(x2\xi, 7/2) not over p(x\, x 2 ) ■ Furthermore, I{X\, X 2 \Yz) in 
(7) in [5] does not hold since (M, Y£~ ) — > (Xii,X 2 i) — > Y-a is not a Markov chain due to unlimited lookahead at the relay. 
If I(Xi, X 2 ; Y3) in (7) in [5] is replaced by I{X\, X 2 , Y 2 \ Y3), then it becomes a valid upper bound. 
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As a simpler example, consider a single-antenna channel given as 

Y 2 = I121X1 + Z 2 

Y3 = h 3 iXi + h S2 X 2 + Z 3 . 

For this causal single-antenna Gaussian relay channel, if 

p > - iv&r) (lh2l]2pi + lh <13) 

then its capacity is given by log{l + (|/i 2 i| 2 + |/i3i| 2 )-Pi}- Note that the above capacity is achieved 
by AF with F = — 2 V _ . In Proposition 9 in [5], they showed a similar result as above, but our 
result is stronger since they require an additional condition \h 2 \\ < \h 32 \ in addition to (13). 

V. Conclusion 

In this paper, we studied a causal discrete memoryless relay network consisting of both causal 
and strictly causal relays. In an analog relay system such as a full duplex relay, if the delay 
spread including the path through relay is much smaller than the inverse of the bandwidth, it 
can be modeled as a causal relay. For this channel, we presented two cut-set bounds, where one 
with messages at causal relays and the other without messages at causal relays. Because we 
considered a general channel model, our bounds cover many known results such as the classical 
cut-set bound. Also, surprisingly, the obtained outer bound can be achieved in some Gaussian 
channels based on AF relaying scheme under some conditions. 
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Appendix A 
Proof of Theorem 1 

For some e n — > as n — > oo, we get 

jes,kes c 

= H(M T ) 

{ = } H(M T \M T c) 

= I(M T] Y£\M T c) + H(M T \Y£, M TC ) 
<I(M T ;Yg c \M T c) + ne n 

n 
n 

i=i 

-HiYs^M.Y^ 1 )] +ne n 

n 

c = ) 5^[jT(r M c, i |M 7 *,3^r 1 ) 
i=i 

-#(y W c i | < M,i*r 1 ) 

+ i/(Fv^|M r c,Fj-- 1 ,^, i ) 
-//(Fv^l-M,^ 1 ,^)] + r*e n , 

n 

{ = } ^ [^(y W c |i |M r c,i^r 1 > Xvc |i ) 

i=l 

-H{Y U o^M,Y^\X Vc ^ 
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(e) 



E 

i=l 



\w\ 



^ {H(Yi j:i \M T c,Y^ c 1 ,X VcA ,Y hA ,Yi 2A , . . . jYi^^i, X ilA , X i2ji , . . . ,X tj _ 



— H(Yi jti \M., Yg c , X V c ti , Yi 1}i , Y i2A , . . . , Yi._ lti , X llA , X l2)i . . . , Xi._ lA )} 

+ H(Y V o ti \M r o, Y£\ Y U e,i, X S e,i) 
-H(Y V c^\M,Yi;7 l i Y U c,i, X S o,i)] + ne n 



(J) 
< 



E 

i=l 



-i^v^l-M,^ 1 , Iwc.i, X 5>i , E w ,i)] + ne n 

n 



i=l 



|W C | 



— i/(lyc j|l^c j, X 5Cji , lw,i)J + ne n 

/ 

^V,i)^[l%-l]\{«i,Z 2 ,-^-i},i' 
y[l:ij-l]\{« 1 ,J 2 ,...,/ J -_i} ) i 



E 



i=i 



\u c \ 

E' 



( 

Xv,i, X[\il j -\]\{h,h,...,lj-- i ),i, 

Yyi-.i^-iWih,...^-!}^ 



E 



\u c \ 

3=1 



\ 



^v c ,«) Q = i 

Mi,ti • • • j y 



+ I{Xs,i, Yu,u Yv c ,i\Yw,i^ X S c ti , Q — i)j + ne r 
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n 



i=i 



/ 



■^V,Q)^[l:i i -l]\{7i 1 ...,i j _i},Q> 



V 



Ji,C 



Y h-u 



/ 

Xv,QiX[i:lj-i\\{h,..;h-i},Q> 



(h) 
< n 



\u c \ 



YL 



V 



Xv c ,Qi 
•^Qi.Qj • • • i ^j-i, 



\ 



+ J(X5,q,F Mi q; Y v =,q|^,q,^s=,q)] + "Cn 

I" 



(0 



n 




-^V^Il^-llU/i,...^-!}) 

Ft 



-<^V C , ^1) ) • • • ) ^-1! 



[l:J J -l]\{Ii,...,Ij-l} 

where Ai is the set of all messages, T = {(j, k) : j E S,k E S c }, Q is a time-sharing 
random variable taking values uniformly in [1 : n] and independent of other variables, (a) is 
because messages are independent, (b) is because of Fano's inequality, (c) is because Y^c j = 
(Yuc t i, Y V c ti ), (d) and (e) are because X V c^ is a function of (Mr^l^r 1 ) and X WCji is a function 
of (M7-C, Y^ c ), (/) is because conditioning reduces entropy, (g) is because of Markov chains 
(M,Y^) {Xv^Xvo^X^^Y^-^^Y^ and (M, Yjr 1 ) 
—> (Xs,i, Xsc,i,Yu,i,Yuc,i) — > Y V c^, (h) is because of Markov chains 

Q — (X V c t Q,X Vt Q, X^-i^Q, Vji :Ji _i] ) Q) — Yi. ; q and Q — (Y^q, Y U c q, X s ^q,X S c q) — Y^q, and 
(i) is by defining X = X Q , Y = Y Q . 
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Appendix B 
Proof of Theorem 2 

For some e n — > as n — > oo, we get 

jev,kes c 

jev,kes c 
= H(M T \M T c) 

= I(M T ; Y^\M TC ) + H{M T \Y£,M T c) 

n 

< I(M T ] Y S c ti \M T c, Y^ 1 ) + ne n 
i=i 

n 

= J2[H(Y S c, i \M TC ,Y*7 1 ) 

i=l 

-HiYso^M^ 1 )] +ne n 

n 
i=l 

- HiYy^M^ 1 ) 

+ H(Y V c ji \M T c,Y£ 1 ,Y Uc , i ) 
-H{Y V o^M,Y^\Y U o^]+ne n 

n 

^Y,[ h ^A m t^y^\x v .,) 

i=l 

— H{Y uci | M. , Yg c 1 , X V cj) 

+ H(Y V c ti \Mrc,Yg7 l ^Yuc^.Xyci) 
-H(Y v . ti \M,Yi;-\ Y U o^ X V c 4 )] + ne n 
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E 

i=i 



\w\ 



+ H(Yyo ii \Mj-o,Y^7 jYuo^, X V c^) 

-H(yvc ti \M, Y£\ Y U c,i, X V c ;i )] + ne n 



(6) 
< 



E 



|W C | 



-H(y V c )i |M, Ejr\ e^, x^, x Vji )] + ne n 



E 



\w\ 



• • i Xi- lt i) 



— ^(^j,il^W c 1 ) ^V c ,i) Ep^-l],ii -X"[l:Zj- l],i) ^V,i)} 
+ -^(^V c ,i|^W c \ E^ c ,«) Xycj) 

—H(Y V c ii \Y^ c 1 ,Y U c^ h X V c jU Y^ 1 ,X Vti )~\ +ne n 
( 

^l:J J -l]\{h,...,«j-i},i 



E 



8=1 



|w c l 

E' 



V 



Yy c 1 , Xyc j, 

■^iijij • • • 3 ^Zj_i,i y 



^(-<^v,i>^w !^v c ,i|^w= ; Ew c ,i>^v c ,i 



(4) 



E 

i=i 



|w c | 

E' 

j'=i 



/ 



E 



V 



Xl lt i, ■ ■ ■ , Xlj_ lt i) 
Eji,i; • • • ) y 
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E 



l" c l 

E' 

3=1 



/ 



V 



-^V,i)^[l%-l]\{Ji,...,i f _i},i, 
y[l:i J -l]\{il,...,i J _i},i 



j E,-,i 



Y" Y 



n 



HXv,i,UuX,Yv c ,i\Uu c ,i,Yu c ,ii Xvci,Q — i)\ + ne r 
( 

X V,Q, ^[l:f J -l]\{ii,...,ij_i} ) Q, 
^1%- l]\{ii,...,Ji-i},Q 



|W C | 

E / 

3=1 



;E, 



y ^i,Q) • ■ • , E,_i,q> <5 y 

+ ^(^V,Q, ^w,q! ^v c ,q|^w c ,q, E^,Q, ^V c ,Q, Q)\ + 



< n 



3=1 



/ 



V 



^V ) Q^[ia j -X]\{h y ..,l j -i},Q- > 
Y[l:l j -l]\{h,...,lj--i},Q 



Xi lt Q, • • • , -Xj wn 



E 



(/) 



/ 

^V,^[l:fj-l]\{7l,.»,*J-l}j 



l" c l 

3=1 



E, 



V 



^Wj-llUIx,...^-!} 



Y h-uQ J 
\ 



Uuc,X V c, 
Xi t , . . . , Xi._ x 
Ei,---,Ej_i J 



-I(X v ,Uu;Y V o\U U o,Y U o,X V c)] + ne n , 



where Ai is the set of all messages, T = {(j, A;) : j E S,k E S c }, Q is a time-sharing 
random variable taking values uniformly in [1 : n] and independent of other variables, (a) is 
because Xyc j is a function of (Mfc, Ey7 x ), (6) is because conditioning reduces entropy, (c) 
is because of Markov chains (M,Y^ 1 ) X V c }i , X V)i , Y^-iiu E jV i and 

(AEEyl 1 ) — )• (EJ7 1 , E^c,i, -Ev c ,i, EJ~\ — >■ Eyc^, where the latter is from Lemma 1, (d) is 
by defining [/, = E i_1 , (e) is because of Markov chains 
Q {Uu c ,q-, Xv c ,q, Xv,q, -^[i%-i],q, Efu.,— i],q) — > Ev,q and 

<3 -> (^4/ c ,q,E/ c ,q,^v c ,q,^v,q, £4/,q) -> E V c iQ , where the latter is from Lemma 2, and (/) is 
by defining X = Xq, Y = Yq, U = U Q . 

Lemma 1: The variables A4, Xy ti , Xyc Yy ti , Y U c U Y v>i , Y v ^ yi , EJ -1 , E^T 1 , E^r 1 form a Markov 
chain as follows, where i E [1 : n], U — SnJ\f , U c — Af \ U = . . . , l\u c \}, h < h if k < h 
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V = S n A/1, and V c = M \ V for 5 c [1 : K}. 

(M.,Yy c ) — >■ (Xy,j, X V c,i, 1^ 1 ,V z }c ) — ► Yv,i, Yw,i) 

Proof: From the induced distribution 

p(m Ml ,x r l,...,x] ( ,y^,...,yl) 



K 



n rbK*) ) n 

yj'eM fc=i 



i=i 



vfceM 



|M>| 

IJ P {Vji I a?M ,i , x [i y _ i] ,i , y [i . j_ i] fi )p( | y) ) 



we get the following relations. 

For j G Ao and i G [1 : n], 

~yi— 1 1 



r, 



y-i— 1 yi— 1 
yi— 1 yi— 1 



i-1 



/ M,xir\x. t 

Y ■■■ 

1 ]ii 



V 



yi-1 V"* -1 

2 W > I U C i 
yi-1 yi-1 

1 V ' J v c 



/ 
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and 



/ 



( 



Y Vi ,Yy 



M,X % U ,Xlf C , 
yi—1 yi— 1 

\ri-l 

1 U ' I U C ■ 
~yi-l yi-1 

1 v ' 1 v c 



yi—1 yi — 1 
V ' ^V c ' 

2 V i 2 V C 



\ 



= o, 

where Af =UVJU C and A/i = V U V c . From above mutual informations, we get for j G Af and 

ie[l: n], 

I (Yj ;i ; M, Yy c 1 |X Vi i, X V c ti , Yy , Y^ c , Yjiy-i],*) 



and 




i-l 



(14) 



Y v ^Y VcX ,M,Y^ 1 



^V,ii ^V c ,i, 

Yu,i, Yuc,ii 

•yi-1 -yi-1 



Y v ,i,Y V c ti ; M,Y VC 



i-l 



Xu,i, Xyc j, 
Xy,i, ^V c ,i) 
Yua, Y U r i 



(15) 



where the first equalities both in (14) and (15) are because Xji e Ao is a function of F? £ J\f . 
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By summing over j G Aq = WU U c , (14) becomes 



IM.I 



i-l 







By combining (15) and (16), we get 



i-l 



-y-j— 1 \ri— 1 
1 U ' 2 W C 



0. 



(16) 



Lemma 2: The variables Q, X VtQ , X V c,q, Y u ,q, Y U o >q , Y v>q , 1v c ,q, U u ,q, U u ^q form a Markov 
chain as follows, where Q is a time-sharing random variable taking values uniformly in [1 : n], 
U = SC\N Q ,U C = N Q \U = {h,...,l m }, k < lj if k< j, V = 5nM, and V c = Ai \ V for 
S C [1 : 

Xv c ,Q,Uu t Q,U U cQ) — >• (Yw,Q, Yw^Yv.Qj^VSq) 

Proof: Similarly as Lemma 1, the following relations hold for j G Aq and i G [1 : n]. 



/ n yQ-l y-Q-1 



Y 



yQ-l y-Q-1 

yO-l yQ-l 
2 V ' V c 



Xv,Q, X V cq, 
X[l:j-1],Q, 



( 



•M,Q, Xy 1 , X uc 1 , 



y-Q-1 yQ-1 
A V ' ^V c ' 

V Q-1 V Q-1 

y-Q-1 y-Q-1 
1 V ' V c 



Xu,Q, Xuc,Q, 
Xv,Q, X V c q, 

Yu,q, Y U c t q 



\ 
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From above mutual informations, we get 

I {Yj,Q]Q\X V ,Q,X V CQ : Yy 1 ,Yy c 1 ,y[ 1 . J -_ 1 ] j Q 



^l:j-l],Q,^l:j-l],Q 



(17) 



and 



/ 



V 



^W,Q> Yu c ,Q, 

yQ-1 y-Q-1 

1 U ' 1 U C J 



Yv,o, Yv c ,a', Q 









i Xv c ,Q 


Yu,Q 


) Yw,q 



Q ) 



(18) 



where the first equalities both in (17) and (18) are because X, q g Ao is a function of Y® G Ao- 
By summing over j G Ao = U UU C , (17) becomes 



/(iv ,q;Q| Xv,o,x V c 1 Q,y z ?- 1 ,y£ 

^ fy^.Q) y U cq] q 



= 



By combining (18) and (19) and by defining Uq = Y Q 1 , we get 

I Yu,q-,Yw=,QiYv,q-,Yv c ,q'iQ 



Xv,Q, Xyo Q, 



Uu,Q, Uuc,Q 



0. 



(19) 
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